Cooling electronic devices in a data center

ABSTRACT

A data center cooling system includes a thermosiphon, an actuator coupled to the thermosiphon, and a controller. The thermosiphon includes an evaporator; a condenser; and at least one conduit coupled between the evaporator and the condenser to transport a working fluid between the evaporator and the condenser. The controller is coupled to the actuator and configured to operate the actuator to adjust a liquid level of the working fluid in the evaporator based, at least in part, on a parameter associated with a heat load of one or more data center heat generating computing devices.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of, and claims priority under 35U.S.C. § 120 to, U.S. patent application Ser. No. 14/494,216, filed Sep.23, 2014, and entitled “Cooling Electronic Devices in a Data Center,”the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

This document relates to systems and methods for providing cooling toelectronic equipment, such as computer server racks and relatedequipment in computer data centers, with a thermosiphon.

BACKGROUND

Computer users often focus on the speed of computer microprocessors(e.g., megahertz and gigahertz). Many forget that this speed often comeswith a cost—higher power consumption. This power consumption alsogenerates heat. That is because, by simple laws of physics, all thepower has to go somewhere, and that somewhere is, in the end, conversioninto heat. A pair of microprocessors mounted on a single motherboard candraw hundreds of watts or more of power. Multiply that figure by severalthousand (or tens of thousands) to account for the many computers in alarge data center, and one can readily appreciate the amount of heatthat can be generated. The effects of power consumed by the criticalload in the data center are often compounded when one incorporates allof the ancillary equipment required to support the critical load.

Many techniques may be used to cool electronic devices (e.g.,processors, memories, networking devices, and other heat generatingdevices) that are located on a server or network rack tray. Forinstance, forced convection may be created by providing a coolingairflow over the devices. Fans located near the devices, fans located incomputer server rooms, and/or fans located in ductwork in fluidcommunication with the air surrounding the electronic devices, may forcethe cooling airflow over the tray containing the devices. In someinstances, one or more components or devices on a server tray may belocated in a difficult-to-cool area of the tray; for example, an areawhere forced convection is not particularly effective or not available.

The consequence of inadequate and/or insufficient cooling may be thefailure of one or more electronic devices on the tray due to atemperature of the device exceeding a maximum rated temperature. Whilecertain redundancies may be built into a computer data center, a serverrack, and even individual trays, the failure of devices due tooverheating can come at a great cost in terms of speed, efficiency, andexpense.

Thermosiphons are heat exchangers that operate using a fluid thatundergoes a phase change. A liquid form of the fluid is vaporized in anevaporator, and heat is carried by the vapor form of the fluid from theevaporator to a condenser. In the condenser, the vapor condenses, andthe liquid form of the fluid is then returned via gravity to theevaporator. Thus, the fluid circulates between the evaporator and thecondenser without need of a mechanical pump.

SUMMARY

This disclosure describes a cooling system, for example, for rackmounted electronic devices (e.g., servers, processors, memory,networking devices or otherwise) in a data center. In various disclosedimplementations, the cooling system includes a thermosiphon system thatincludes a condenser, evaporator, and conduit fluidly coupledtherebetween. The thermosiphon system is thermally coupled to theelectronic devices such that heat generated by such devices istransferred to a working fluid in the evaporator, vaporizing the workingfluid. The vaporized working fluid moves to the condenser, where itreleases the transferred heat (e.g., to air or airflow around thecondenser) and condenses to a liquid. The thermosiphon system, or a partthereof, is adjustable based on a sensed or measured parameter that isassociated with a heat load or power load of the electronic devices. Insome implementations, one or more components of the thermosiphon systemare adjustable to adjust and/or maintain a liquid level of the workingfluid in the evaporator based, at least in part, on the sensed ormeasured parameter.

In an example implementation, a data center cooling system includes athermosiphon, an actuator coupled to the thermosiphon, and a controller.The thermosiphon includes an evaporator; a condenser; and at least oneconduit coupled between the evaporator and the condenser to transport aworking fluid between the evaporator and the condenser. The controlleris coupled to the actuator and configured to operate the actuator toadjust a liquid level of the working fluid in the evaporator based, atleast in part, on a parameter associated with a heat load of one or moredata center heat generating computing devices.

In a first aspect combinable with the general implementation, theactuator includes a height adjustment assembly coupled to the condenser.

In a second aspect combinable with any of the previous aspects, theheight adjustment assembly is mounted to the condenser and arranged toadjust a position of the condenser to adjust a vertical distance betweenthe condenser and the evaporator based, at least in part, on theparameter.

In a third aspect combinable with any of the previous aspects, theheight adjustment assembly is mounted to the condenser and arranged tovibrate the condenser based, at least in part, on the parameter.

In a fourth aspect combinable with any of the previous aspects, acombination of the controller and the height adjustment assembly isarranged as a bimetallic member in contact with at least one of thecondenser or the conduit, and the bimetallic member is arranged toadjust a position of the condenser to adjust a vertical distance betweenthe condenser and the evaporator based, at least in part, on theparameter. The parameter includes a temperature difference between atemperature of the condenser or the conduit and a reference temperature.

In a fifth aspect combinable with any of the previous aspects, acombination of the controller and the height adjustment assembly isarranged as a phase change motor in contact with the condenser, and thephase change motor is arranged to adjust a position of the condenser toadjust a vertical distance between the condenser and the evaporatorbased, at least in part, on the parameter. The parameter includes atemperature of the condenser relative to a temperature of a phase changematerial of the phase change motor.

In a sixth aspect combinable with any of the previous aspects, theactuator includes a piston mounted in a working volume of the condenser.

In a seventh aspect combinable with any of the previous aspects, thepiston is arranged to oscillate in the condenser to adjust the workingvolume based, at least in part, on the parameter.

In an eighth aspect combinable with any of the previous aspects, thepiston is arranged to vibrate the condenser based, at least in part, onthe parameter.

In a ninth aspect combinable with any of the previous aspects, theactuator includes an angular adjustment assembly coupled to thecondenser.

In a tenth aspect combinable with any of the previous aspects, theangular adjustment assembly is mounted to the condenser and arranged torotate or pivot the condenser based, at least in part, on the parameter.

In an eleventh aspect combinable with any of the previous aspects, theangular adjustment assembly is arranged to vibrate the condenser based,at least in part, on the parameter.

In a twelfth aspect combinable with any of the previous aspects, theconduit includes a liquid line and a vapor line, and the actuatorincludes a valve positioned in the liquid line.

In a thirteenth aspect combinable with any of the previous aspects, thevalve is arranged to modulate toward a closed position or an openposition based, at least in part, on the parameter.

In a fourteenth aspect combinable with any of the previous aspects, theactuator includes a vibration assembly coupled to the condenser andarranged to vibrate the condenser based, at least in part, on theparameter.

In a fifteenth aspect combinable with any of the previous aspects, theconduit couples the evaporator and the condenser at a downward anglerelative to gravity from the condenser to the evaporator.

In a sixteenth aspect combinable with any of the previous aspects, theconduit is flexible.

In a seventeenth aspect combinable with any of the previous aspects, theparameter includes at least one of: a temperature of air adjacent therack-mounted device, a temperature of air adjacent the condenser, atemperature of the one or more data center heat generating computingdevices, a temperature of a motherboard that supports the one or moredata center heat generating computing devices, the liquid level of theworking fluid in the evaporator, a pressure of the working fluid, atemperature of the working fluid, a power usage of the one or more datacenter heat generating computing devices, a frequency of the one or moredata center heat generating computing devices, or a utilization of theone or more data center heat generating computing devices.

An eighteenth aspect combinable with any of the previous aspects furtherincludes a wicking material mounted in the conduit between the condenserand the evaporator.

In a nineteenth aspect combinable with any of the previous aspects, theevaporator includes a base and a case that define a chamber for theworking fluid; and a plurality of fins integrally formed with the basethat extend into the chamber from the base.

In another generation implementation, a method for cooling heatgenerating devices in a data center includes circulating a working fluidbetween an evaporator of a thermosiphon and a condenser of thethermosiphon in a downwardly angled conduit that fluidly couples theevaporator and the condenser, where the working fluid includes a gaswhen circulated from the evaporator to the condenser and a liquid whencirculated from the condenser to the evaporator; cooling, based on thecirculating, one or more heat generating devices in thermalcommunication with the evaporator; determining a parameter associatedwith a heat load of at least one of the heat generating devices; andbased at least in part on the measured parameter, operating an actuatorcoupled to the thermosiphon to adjust a liquid level of the workingfluid in the evaporator.

In a first aspect combinable with the general implementation, operatingan actuator includes operating a height adjustment assembly coupled tothe condenser to adjust a position of the condenser to adjust a verticaldistance between the condenser and the evaporator based, at least inpart, on the determined parameter.

In a second aspect combinable with any of the previous aspects,operating the actuator further includes operating the height adjustmentassembly to vibrate the condenser based, at least in part, on thedetermined parameter.

In a third aspect combinable with any of the previous aspects, operatingan actuator includes adjusting a position of the condenser with abimetallic member in contact with at least one of the condenser or theconduit to adjust a vertical distance between the condenser and theevaporator based, at least in part, on the determined parameter. Thedetermined parameter includes a temperature difference between atemperature of the condenser or the conduit and a reference temperature.

In a fourth aspect combinable with any of the previous aspects,operating an actuator includes adjusting a position of the condenserwith a phase change motor in contact with the condenser, the phasechange motor arranged to adjust a position of the condenser to adjust avertical distance between the condenser and the evaporator based, atleast in part, on the determined parameter. The determined parameterincludes a temperature of the condenser relative to a temperature of aphase change material of the phase change motor.

In a fifth aspect combinable with any of the previous aspects, operatingan actuator includes moving a piston mounted in a working volume of thecondenser to adjust the working volume based, at least in part, on thedetermined parameter.

In a sixth aspect combinable with any of the previous aspects, operatingthe actuator further includes vibrating the condenser with the pistonbased, at least in part, on the determined parameter.

In a seventh aspect combinable with any of the previous aspects,operating an actuator includes rotating or pivoting the condenser withan angular adjustment assembly coupled to the condenser based, at leastin part, on the determined parameter.

In an eighth aspect combinable with any of the previous aspects,operating the actuator further includes vibrating the condenser with theangular adjustment assembly based, at least in part, on the determinedparameter.

In a ninth aspect combinable with any of the previous aspects, theconduit includes a liquid line and a vapor line, and the actuatorincludes a valve positioned in the liquid line.

In a tenth aspect combinable with any of the previous aspects, operatingan actuator includes modulating the valve based, at least in part, onthe determined parameter.

In an eleventh aspect combinable with any of the previous aspects,operating an actuator includes vibrating the condenser based, at leastin part, on the determined parameter.

In a twelfth aspect combinable with any of the previous aspects, theparameter includes at least one of: a temperature of air adjacent therack-mounted device, a temperature of air adjacent the condenser, atemperature of the heat generating device, a temperature of a surfacethat supports the heat generating device, the liquid level of theworking fluid in the evaporator, a pressure of the working fluid, atemperature of the working fluid, a power usage of the heat generatingdevice, a frequency of the heat generating device, or a utilization ofthe heat generating device.

In another general implementation, a thermosiphon cooling system for arack-mounted device in a data center includes an evaporator; a condenserfluidly coupled to the evaporator with a fluid pathway that slopesdownward from the condenser to the evaporator; a working fluid enclosedwithin the evaporator, the condenser, and the fluid pathway; means fordetermining at least one parameter associated with an amount of heatgenerated by the rack-mounted device; and means for adjusting a liquidlevel of the working fluid in the evaporator based, at least in part, onthe parameter associated with the amount of heat generated by therack-mounted device.

In a first aspect combinable with the general implementation, the meansfor adjusting a liquid level of the working fluid in the evaporatorincludes means for vibrating the condenser.

In a second aspect combinable with any of the previous aspects, themeans for adjusting a liquid level of the working fluid in theevaporator includes means for adjusting a vertical distance between thecondenser and the evaporator.

In a third aspect combinable with any of the previous aspects, the meansfor adjusting a liquid level of the working fluid in the evaporatorincludes means for adjusting a rate of liquid flowing in the fluidpathway from the condenser to the evaporator.

In a fourth aspect combinable with any of the previous aspects, theparameter includes at least one of: a temperature of air adjacent therack-mounted device, a temperature of air adjacent the condenser, atemperature of the rack-mounted device, a temperature of a surface thatsupports the rack-mounted device, the liquid level of the working fluidin the evaporator, a pressure of the working fluid, a temperature of theworking fluid, the power usage of the rack-mounted device, a frequencyof the rack-mounted device, or a utilization of the rack-mounted device.

A fifth aspect combinable with any of the previous aspects furtherincludes a fan positioned to circulate an airflow over the condenser.

A sixth aspect combinable with any of the previous aspects furtherincludes one or more heat transfer surfaces mounted to the condenser.

In another general implementation, a server tray sub-assembly includes amotherboard; a plurality of heat generating electronic devices mountedon the motherboard; a thermosiphon mounted on the motherboard; and acontrol system. The thermosiphon includes an evaporator in heat transfercommunication with the plurality of heat generating electronic devices;a condenser fluidly coupled to the evaporator with a fluid conduit thatslopes downward from the condenser to the evaporator; and a multi-phasefluid contained in the thermosiphon. The control system includes asensing device operable to sense a value associated with an amount ofheat generated by the plurality of heat generating electronic devices;and an actuator operatively coupled to the thermosiphon to adjust anamount of the multi-phase fluid in the evaporator based, at least inpart, on the sensed value.

In a first aspect combinable with the general implementation, theactuator is operatively coupled to the condenser and is configured toadjust the condenser based, at least in part, on the sensed value.

In a second aspect combinable with any of the previous aspects, theactuator is configured to adjust at least one of a vertical distancebetween the condenser and the evaporator; a working volume of thecondenser; or an angular position of the condenser relative to themotherboard.

In a third aspect combinable with any of the previous aspects, theactuator is configured to vibrate the condenser.

In a fourth aspect combinable with any of the previous aspects, theactuator is configured to adjust a flow of a liquid phase of themulti-phase fluid from the condenser to the evaporator based, at leastin part, on the sensed value.

In a fifth aspect combinable with any of the previous aspects, theactuator includes a valve arranged in a liquid line of the fluidconduit.

In a sixth aspect combinable with any of the previous aspects, theactuator includes a wicking material mounted in a liquid line of thefluid conduit.

In a seventh aspect combinable with any of the previous aspects, theconduit is flexible.

In an eighth aspect combinable with any of the previous aspects, thesensed value includes at least one of: a temperature of air adjacent theplurality of heat generating electronic devices, a temperature of airadjacent the condenser, a temperature of the plurality of heatgenerating electronic devices, a temperature of the motherboard, aliquid level of the multi-phase fluid in the evaporator, a pressure ofthe multi-phase fluid, a temperature of the multi-phase fluid, a powerusage of the plurality of heat generating electronic devices, afrequency of one or more of the plurality of heat generating electronicdevices, or a utilization of one or more of the plurality of heatgenerating electronic devices.

In another general implementation, a thermosiphon cooling system for arack-mounted device in a data center includes an evaporator; a condenserfluidly coupled to the evaporator with a flexible fluid pathway thatslopes downward from the condenser to the evaporator; and a workingfluid enclosed within the evaporator, the condenser, and the fluidpathway. The evaporator includes a base; a case attachable to the base,the base and the case defining a chamber for a working fluid; and a heattransfer surface integrally formed with the base and extending from thebase into the chamber.

In a first aspect combinable with the general implementation, the heattransfer surface includes a plurality of fins integrally formed with thebase.

In a second aspect combinable with any of the previous aspects, the finsare sized to extend into the chamber above a liquid level of the workingfluid.

In a third aspect combinable with any of the previous aspects, the finsare formed in substantially parallel rows.

In a fourth aspect combinable with any of the previous aspects, each ofthe fins comprise a textured outer surface.

In a fifth aspect combinable with any of the previous aspects, thetextured outer surface is configured to facilitate a capillary effect ofthe working fluid.

In a sixth aspect combinable with any of the previous aspects, the basecomprises a pan to hold liquid working fluid.

Various implementations of a data center cooling system according to thepresent disclosure may include one, some, or all of the followingfeatures. For example, the thermosiphon cooling system may provide for amore flexible and adaptable cooling system in response to changingcooling requirements of a server tray, components on the server tray(e.g., CPU, memory, or otherwise), a networking tray, or other rackmounted system in a data center. For example, the thermosiphon systemmay more accurately match the cooling requirement, thereby increasingefficiency of the cooling system while minimizing overcooling of certaincomponents. In some example implementations, the thermosiphon coolingsystem may have an improved efficiency at lower required cooling powers(e.g., lower heat generated by the components). The thermosiphon coolingsystem may also have an increased cooling capacity at higher requiredcooling powers (e.g., higher heat generated by the components on theserver tray). As another example, the thermosiphon cooling system mayprovide flexibility for us in multiple generations of heat generatingcomponents, while using the same parts and manufacturing process.

Various implementations of a data center cooling system according to thepresent disclosure may include one, some, or all of the followingfeatures. The thermosiphon system can fit within the limited horizontaland vertical space of the server rack. A thin layer of liquid can bemaintained in the evaporator over the region where the evaporatorcontacts the electronic device, thus reducing thermal resistance of theevaporator to absorption of heat from the electronic device. Inaddition, the likelihood of flooding of this region can be reduced, thusreducing the likelihood of failure of the thermosiphon system due toincreased thermal resistance.

The details of one or more embodiments are set forth in the accompanyingdrawings and the description below. Other features, objects, andadvantages will be apparent from the description and drawings, and fromthe claims.

DESCRIPTION OF DRAWINGS

FIG. 1 illustrates a side view of a server rack and a server-racksub-assembly configured to mount within a rack used in a data centerenvironment;

FIGS. 2A-2B illustrate schematic side and top views, respectively, of aserver rack sub-assembly that includes an example implementation of athermosiphon cooling system;

FIGS. 3A-3B illustrate schematic side and top views, respectively, of aserver rack sub-assembly that includes another example implementation ofa thermosiphon cooling system;

FIGS. 4A-4B illustrate schematic side and top views, respectively, of aserver rack sub-assembly that includes another example implementation ofa thermosiphon cooling system;

FIG. 5 illustrates a schematic side view of a server rack sub-assemblythat includes another example implementation of a thermosiphon coolingsystem;

FIGS. 6A-6B illustrate schematic side and top views of a portion of athermosiphon cooling system;

FIGS. 7-8 are flowcharts that illustrate example methods of cooling heatgenerating devices in a data center with a thermosiphon cooling system;and

FIG. 9 is a schematic diagram of a computer system that can be used forthe operations described in association with any of thecomputer-implemented methods described herein.

DETAILED DESCRIPTION

This document discusses a thermosiphon system that can be implemented toremove heat from an electronic device, e.g., a component of computingequipment, such as a processor or memory. The evaporator of thethermosiphon system contacts the electronic device so that theelectronic device experiences a conductive heat transfer effect. Thus,the thermosiphon system can act as a heat sink for the electronicdevice, reducing the likelihood of overheating and subsequent failure ofthe electronic device. The thermosiphon system can be mounted on orintegrated with a server rack sub-assembly for insertion into a serverrack. The server rack sub-assembly can contain or support a number ofheat-generating electronic devices, and the evaporator of thethermosiphon system can contact one or more of the electronic devices.In addition, the thermosiphon system can be mounted on a circuit cardassembly, a daughter card, and/or other boards that carryheat-generating electronic devices.

In some example implementations, one or more components of thethermosiphon system can be adjusted (e.g., dynamically during operation)to adjust a liquid level of the working fluid in the evaporator to, forinstance, better match a heat load generated by the electronic device.In some aspects, by matching a dynamic heat load of the electronicdevice, the thermosiphon system may be operated more efficiently. Forexample, the thermosiphon system may operate most efficiently at a“dryout” condition, where all or substantially all of the liquid workingfluid in the evaporator is vaporized by heat transfer to the evaporatorfrom the electronic device. Since the electronic device may not output aconstant heat power (e.g., due to changes in operational speed,frequency, utilization, or otherwise), the thermosiphon system that isdynamically adjusted to operate at the dryout condition (or otherdesirable operating condition) may be more efficient.

FIG. 1 illustrates an example system 100 that includes a server rack105, e.g., a 13 inch or 19 inch server rack, and multiple server racksub-assemblies 110 mounted within the rack 105. Although a single serverrack 105 is illustrated, server rack 105 may be one of a number ofserver racks within the system 100, which may include a server farm or aco-location facility that contains various rack mounted computersystems. Also, although multiple server rack sub-assemblies 110 areillustrated as mounted within the rack 105, there might be only a singleserver rack sub-assembly. Generally, the server rack 105 definesmultiple slots 107 that are arranged in an orderly and repeating fashionwithin the server rack 105, and each slot 107 is a space in the rackinto which a corresponding server rack sub-assembly 110 can be placedand removed. For example, the server rack sub-assembly can be supportedon rails 112 that project from opposite sides of the rack 105, and whichcan define the position of the slots 107.

The slots 107, and the server rack sub-assemblies 110, can be orientedwith the illustrated horizontal arrangement (with respect to gravity).Alternatively, the slots 107, and the server rack sub-assemblies 110,can be oriented vertically (with respect to gravity), although thiswould require some reconfiguration of the evaporator and condenserstructures described below. Where the slots are oriented horizontally,they may be stacked vertically in the rack 105, and where the slots areoriented vertically, they may be stacked horizontally in the rack 105.

Server rack 105, as part of a larger data center for instance, mayprovide data processing and storage capacity. In operation, a datacenter may be connected to a network, and may receive and respond tovarious requests from the network to retrieve, process, and/or storedata. In operation, for example, the server rack 105 typicallyfacilitates the communication of information over a network with userinterfaces generated by web browser applications of users who requestservices provided by applications running on computers in thedatacenter. For example, the server rack 105 may provide or help providea user who is using a web browser to access web sites on the Internet orthe World Wide Web.

The server rack sub-assembly 110 may be one of a variety of structuresthat can be mounted in a server rack. For example, in someimplementations, the server rack sub-assembly 110 may be a “tray” ortray assembly that can be slidably inserted into the server rack 105.The term “tray” is not limited to any particular arrangement, butinstead applies to motherboard or other relatively flat structuresappurtenant to a motherboard for supporting the motherboard in positionin a rack structure. In some implementations, the server racksub-assembly 110 may be a server chassis, or server container (e.g.,server box). In some implementations, the server rack sub-assembly 110may be a hard drive cage.

Referring to FIGS. 2A-2B, the server rack sub-assembly 110 includes aframe or cage 120, a printed circuit board 122, e.g., motherboard,supported on the frame 120, one or more heat-generating electronicdevices 124, e.g., a processor or memory, mounted on the printed circuitboard 122, and a thermosiphon system 130. One or more fans 126 can alsobe mounted on the frame 120.

The frame 120 can include or simply be a flat structure onto which themotherboard 122 can be placed and mounted, so that the frame 120 can begrasped by technicians for moving the motherboard into place and holdingit in position within the rack 105. For example, the server racksub-assembly 110 may be mounted horizontally in the server rack 105 suchas by sliding the frame 120 into the slot 107 and over a pair of railsin the rack 105 on opposed sides of the server rack sub-assembly110—much like sliding a lunch tray into a cafeteria rack. Although FIGS.2A-2B illustrate the frame 120 extending below the motherboard 122, theframe can have other forms (e.g., by implementing it as a peripheralframe around the motherboard) or may be eliminated so that themotherboard itself is located in, e.g., slidably engages, the rack 105.In addition, although FIG. 2A illustrates the frame 120 as a flat plate,the frame 120 can include one or more side walls that project upwardlyfrom the edges of the flat plate, and the flat plate could be the floorof a closed-top or open-top box or cage.

The illustrated server rack sub-assembly 110 includes a printed circuitboard 122, e.g., a motherboard, on which a variety of components aremounted, including heat-generating electronic devices 124. Although onemotherboard 122 is illustrated as mounted on the frame 120, multiplemotherboards may be mounted on the frame 120, depending on the needs ofthe particular application. In some implementations, the one or morefans 126 can be placed on the frame 120 so that air enters at the frontedge (at the left hand side in FIGS. 2A-2B) of the server racksub-assembly 110, closer to the front of the rack 105 when thesub-assembly 110 is installed in the rack 105, flows (as illustrated)over the motherboard 122, over some of the heat generating components onthe motherboard 122, and is exhausted from the server rack assembly 110at the back edge (at the right hand side), closer to the back of therack 105 when the sub-assembly 110 is installed in the rack 105. The oneor more fans 126 can be secured to the frame 120 by brackets. Thus, thefans 126 can pull air from within the frame 120 area and push the airafter it has been warmed out the rack 105. An underside of themotherboard 122 can be separated from the frame 120 by a gap.

The thermosiphon system 130 includes an evaporator 132, a condenser 134,and condensate/vapor lines 136 connecting the evaporator 132 to thecondenser 134. The evaporator 132 contacts the electronic device 124 sothat heat is drawn by conductive heat transfer from the electronicdevice 124 to the evaporator 132. For example, the evaporator 132 is inconductive thermal contact with the electronic device 124. Inparticular, the bottom of the evaporator 132 contacts the top of theelectronic device 124. In operation, heat from the electronic device 124causes a working fluid 148 in the evaporator 132 to evaporate. The vaporthen passes through condensate/vapor lines 136 to the condenser 134.Heat is radiated away from the condenser 134, e.g., into air around thecondenser 134 or into air blown or drawn by the one or more fans 126that passes across the condenser 134, causing the working fluid 148 tocondense. As shown in FIG. 2A, the condenser 134 can be located betweenthe one or more fans 126 from the evaporator 132, but could also belocated on an opposite side of one or more of fans 126 (e.g., near anedge of the sub-assembly 110).

As shown in FIG. 2A, the condensate/vapor line 136 is at a slight(non-zero) angle so that gravity causes the condensed working fluid 148to flow back through the condensate/vapor line 136 to the evaporator132. Thus, in some implementations, at least a portion of thecondensate/vapor lines 136 is not parallel to the main surface of theframe 120. For example, the condenser-side end of the condensate/vaporline 136 can be about 1-5 mm, e.g., 2 mm, above the evaporator-side endof the condensate/vapor line 136. However, it is also possible for thecondensate/vapor line 136 to be horizontal tube, or even at a slightlynegative angle (although the positive angle provides an advantage ofgravity improving flow of the liquid from the condenser to theevaporator). Because there can be multiple heat generating electronicdevices on a single motherboard, there can be multiple evaporators onthe motherboard, where each evaporator corresponds to a singleelectronic device. As shown in FIG. 2A, there is a first evaporator 132and a second evaporator 132 as well as a first electronic device 124 anda second electronic device 124. The condensate/vapor line 136 connectingthe first evaporator to the second evaporator can be level.

During operation, the top surface of the working fluid 148 (as a liquid)inside the condenser 134 will be above the top surface liquid height ofthe working fluid 148 in the evaporator 132, e.g., by 1 to 10 mm. It canbe easier to achieve this with a condensate/vapor line 136 that is at aslight (positive non-zero) angle, but proper selection of the thermaland mechanical properties of the working fluid 148 in view of theexpected heat transport requirements for the thermosiphon system 130 maystill achieve this for a condensate/vapor line 136 that is horizontal orat a slightly negative angle. During operation, the liquid phase of aworking fluid 148 can fill a bottom portion of an interior volume of thecondensate/vapor line 136, with the bottom portion extending from thecondenser 134 to the evaporator 132, and a vapor phase of the workingfluid 148 can pass through a top portion of the interior volume of thecondensate/vapor line 136, with the top portion extending from thecondenser 134 to the evaporator 132.

In some implementations, the condenser 134 can be located at a heightabove the evaporator 132 such that a liquid phase of the working fluid148 fills a bottom portion of an interior volume of the condensate/vaporline 136 and such that during operation a top surface of the liquidphase has at a non-zero angle relative to horizontal from the condenser132 to the evaporator 134, and a vapor phase of the working fluid 148can pass through a top portion of the interior volume of thecondensate/vapor line 136, the top portion extending from the condenser134 to the evaporator 132.

FIGS. 2A-2B illustrate a thermosiphon system 130 with multipleevaporators 132; each evaporator 132 can contact a different electronicdevice 124, or multiple evaporators 132 could contact the sameelectronic device, e.g., if the electronic device is particularly largeor has multiple heat generating regions. The multiple evaporators 132can be connected by the condensate/vapor lines 136 to the condenser 134in series, e.g., a first condensate/vapor line connects the condenser134 to a first evaporator 132, and a second condensate/vapor line 136connects the first evaporator 132 to a second evaporator 132.Alternatively, some or all of the multiple evaporators 132 can beconnected by the condensate/vapor lines 136 to the condenser 134 inparallel, e.g., a first condensate/vapor line connects the condenser toa first evaporator, and a second condensate/vapor line connects thecondenser 134 to a second evaporator. Advantages of a serialimplementation may be fewer tubes, whereas an advantage of paralleltubes is that the tube diameters can be smaller.

FIGS. 2A-2B illustrate a thermosiphon system 130 in which a common lineis used for both the condensate flow from the condenser 134 to theevaporator 132 and for vapor flow from the evaporator 132 to thecondenser 134. Thus, in this implementation the fluid coupling betweenthe evaporator 132 and the condenser 134 consists of the combinedcondensate and vapor transfer line 136. In some implementations, therecan be separate lines for the vapor and the condensate. However, apotential advantage of the combined condensate and vapor transfer lineis that the line can be connected to a side of the condenser, reducingthe vertical height of the system relative to a system with a separateline for the vapor, since the vapor line is typically coupled to or nearthe top of the evaporator. The condensate/vapor line 136 can be aflexible tube or pipe, e.g., of copper or aluminum.

As shown in FIGS. 2A-2B, a controller 144 (or control system) iscommunicably coupled to one or more temperature sensors 146, one or morepressure/liquid level sensors 150 located in the evaporator 132, anactuator 142 mounted between the condenser 134 and a base 140 (which, insome aspects, can be removed and the actuator 142 can be mounted to theframe 120), as well as fans 126 (e.g., to control a speed or state ofthe fans 126). Generally, the controller 144 may receive one or moreinputs from the sensors 146 and/or sensors 150 (as well as other inputs)and control the actuator 142 to adjust a position of the condenser 134to, for example, better match a cooling capacity of the thermosiphonsystem 130 with a heat load of the electronic devices 124.

In some aspects, inputs into the controller 144, such as the sensors 146and/or sensors 150 may be indicative of the heat load of the electronicdevices 124. For example, the sensors 146 may measure a temperature ofthe electronic devices 124 and/or the motherboard 122. Also, sensors 150may measure a temperature, pressure, and/or liquid level of the workingfluid 148 in the evaporator 132. Although not specifically shown,temperature, pressure, and/or level of the working fluid 148 may bemeasured in the condenser 134 or conduit 136 as indicative of the heatload of the electronic devices 124.

One or more operational parameters of the electronic devices 124 mayalso be measured by sensors (not shown) that may be indicative of theheat load of the electronic devices 124. For example, power usage (e.g.,current, voltage, or power) of the electronic devices 124 may bemeasured and may be indicative of the heat load of the electronicdevices 124. As another example, operational speed or frequency (e.g.,Hz) of the electronic devices 124 may be measured and may be indicativeof the heat load of the electronic devices 124. As another example,utilization (e.g., number of jobs executed or to be executed, orotherwise) of the electronic devices 124 may be measured and may beindicative of the heat load of the electronic devices 124. Suchparameters may also be provided to the controller 144 and used to adjustthe actuator 142.

The actuator 142 may be adjusted by the controller 144 based, at leastin part on the measured or sensed parameters described above. In someimplementations, the actuator 142 may adjust a height of the condenser134 above the frame 120, which may also adjust a relative verticaldistance between the condenser 134 and the evaporator 132. In someexamples, as the relative vertical distance between the condenser 134and the evaporator 132 increases, more liquid working fluid 148 flows tothe evaporator 132, thereby increasing a cooling capacity of thethermosiphon system (e.g., allowing a liquid level of the working fluid148 to increase in the evaporator 132). As the relative verticaldistance between the condenser 134 and the evaporator 132 decreases,less liquid working fluid 148 flows to the evaporator 132, therebydecreasing a cooling capacity of the thermosiphon system (e.g., allowinga liquid level of the working fluid 148 to decrease in the evaporator132). Although the cooling capacity of the thermosiphon system may bedecreased, the cooling efficiency is increased by, for instance,preventing or reducing a back-up of liquid working fluid 148 in theevaporator 132.

By adjusting the relative vertical distance between the evaporator 132and the condenser 134 (thereby adjusting a liquid level of working fluid148 in the evaporator 132), the cooling capacity of the thermosiphonsystem 130 may more closely match the heat load of the electronicdevices 124 (e.g., indicated by the one or more sensed or measureparameters described above). By matching or closely matching the heatload, the thermosiphon system 130 may operate more efficiently, forexample, operate closer to a dryout condition where all or most of theliquid working fluid 148 in the evaporator 132 is vaporized by the heatof the electronic devices 124.

In some implementations, the actuator 142 may adjust an angular positionof the condenser 134 relative to the frame 120, for example, by rotatingand/or pivoting the condenser 134. In some examples, as the condenser134 is rotated or pivoted, more liquid working fluid 148 may flow to theevaporator 132, thereby increasing a cooling capacity of thethermosiphon system (e.g., allowing a liquid level of the working fluid148 to increase in the evaporator 132). For example, by rotating thecondenser 134 relative to the evaporator 132, the pressure stackup maybe changed. Further, in some aspects, the condenser 134 may be rotatedas well as height-adjusted to adjust an amount of liquid working fluid148 that flows from the condenser 134 to the evaporator 132. In someaspects, rotation of the condenser 134 may be preferred to adjusting theheight of the condenser 134 because, for instance, the vertical heightadjustment may be constrained by certain degrees of freedom (e.g., dueto space allowances in a rack or on a server tray sub-assembly). Asdescribed above, by adjusting a liquid level of working fluid 148 in theevaporator 132, the cooling capacity of the thermosiphon system 130 maymore closely match the heat load of the electronic devices 124 (e.g.,indicated by the one or more sensed or measure parameters describedabove).

In some implementations, the actuator 142 may adjust a vibratory stateof the condenser (in addition to or alternatively with adjusting aheight or angular position). For example, based on a command from thecontroller 144, the actuator 142 may vibrate the condenser 134 in orderto, for instance, minimize a size of bubbles in the working fluid 148enclosed within the condenser 134. By minimizing a size of the bubblesof the working fluid 148 (e.g., breaking up larger bubbles into smallerbubbles), a thermal resistance of the working fluid 148 is decreased inthe condenser 134 (e.g., to condensing) and/or evaporator (e.g., toboiling). As the thermal resistance to boiling/condensing is decreased,a heat transfer coefficient of the working fluid 148 is increased,thereby increasing a cooling capacity of the thermosiphon system 130.Thus, vibration of the condenser 134 by the actuator 142 (or of theevaporator 132 in alternative implementations) may adjust a coolingcapacity of the thermosiphon system 130 to match or more closely match aheat load of the electronic devices 124.

In some implementations of the illustrated thermosiphon system 130, theactuator 142 is a mechanical or electro-mechanical device (e.g.,piston-cylinder, motor, or otherwise) that receives commands from thecontroller 144 to adjust the condenser 134 as described above. Inanother implementation, the controller 144/actuator 142 combination maybe implemented as a bimetallic strip or member 142 that is contactinglyengaged with the condenser 134 (or conduit 136) and the evaporator 132(or the electronic devices 124, motherboard 122 or frame 120). Based ona temperature difference between the components contacted by thebimetallic member 142, the member 142 may contract or expand to changethe height difference between the condenser 134 and the evaporator 132to adjust a cooling capacity of the thermosiphon system 130 as describedabove. In this implementation, no external power and/or sensors (e.g.,input to the controller 144) may be necessary for operation of theadjustable thermosiphon system 130, thereby decreasing complexity of thesystem 130.

In other example implementations, the controller 144/actuator 142combination may be implemented as a phase change motor or phase changelinear actuator (e.g., wax motor) that is contactingly engaged with thecondenser 134 and the frame 120. Based on a temperature differencebetween the components contacted by the phase change motor (e.g., thecondenser 134 and the frame 120), the phase change motor may adjust theheight difference between the condenser 134 and the evaporator 132 toadjust a cooling capacity of the thermosiphon system 130 as describedabove. As with the bimetallic strip implementation, no external powerand/or sensors (e.g., input to the controller 144) may be necessary foroperation of the adjustable thermosiphon system 130 with a phase changemotor, thereby decreasing complexity of the system 130.

FIGS. 3A-3B illustrate schematic side and top views, respectively, of aserver rack sub-assembly 210 that includes another exampleimplementation of a thermosiphon cooling system 230. The server racksub-assembly 210 includes a frame or cage 220, a printed circuit board222, e.g., motherboard, supported on the frame 220, one or moreheat-generating electronic devices 224, e.g., a processor or memory,mounted on the printed circuit board 222, and a thermosiphon system 230.One or more fans 226 can also be mounted on the frame 220.

The frame 220 can include or simply be a flat structure onto which themotherboard 222 can be placed and mounted, so that the frame 220 can begrasped by technicians for moving the motherboard into place and holdingit in position within the rack 105. For example, the server racksub-assembly 210 may be mounted horizontally in the server rack 105 suchas by sliding the frame 220 into the slot 107 and over a pair of railsin the rack 105 on opposed sides of the server rack sub-assembly210—much like sliding a lunch tray into a cafeteria rack. Although FIGS.3A-3B illustrate the frame 220 extending below the motherboard 222, theframe can have other forms (e.g., by implementing it as a peripheralframe around the motherboard) or may be eliminated so that themotherboard itself is located in, e.g., slidably engages, the rack 105.In addition, although FIG. 3A illustrates the frame 220 as a flat plate,the frame 220 can include one or more side walls that project upwardlyfrom the edges of the flat plate, and the flat plate could be the floorof a closed-top or open-top box or cage.

The illustrated server rack sub-assembly 210 includes a printed circuitboard 222, e.g., a motherboard, on which a variety of components aremounted, including heat-generating electronic devices 224. Although onemotherboard 222 is illustrated as mounted on the frame 220, multiplemotherboards may be mounted on the frame 220, depending on the needs ofthe particular application. In some implementations, the one or morefans 226 can be placed on the frame 220 so that air enters at the frontedge (at the left hand side in FIGS. 3A-3B) of the server racksub-assembly 210, closer to the front of the rack 105 when thesub-assembly 210 is installed in the rack 105, flows (as illustrated)over the motherboard 222, over some of the heat generating components onthe motherboard 222, and is exhausted from the server rack assembly 210at the back edge (at the right hand side), closer to the back of therack 105 when the sub-assembly 210 is installed in the rack 105. The oneor more fans 226 can be secured to the frame 220 by brackets. Thus, thefans 226 can pull air from within the frame 220 area and push the airafter it has been warmed out the rack 105. An underside of themotherboard 222 can be separated from the frame 220 by a gap.

The thermosiphon system 230 includes an evaporator 232, a condenser 234,and condensate/vapor lines 236 connecting the evaporator 232 to thecondenser 234. The evaporator 232 contacts the electronic device 224 sothat heat is drawn by conductive heat transfer from the electronicdevice 224 to the evaporator 232. For example, the evaporator 232 is inconductive thermal contact with the electronic device 224. Inparticular, the bottom of the evaporator 232 contacts the top of theelectronic device 224. In operation, heat from the electronic device 224causes a working fluid 248 in the evaporator 232 to evaporate. The vaporthen passes through condensate/vapor lines 236 to the condenser 234.Heat is radiated away from the condenser 234, e.g., into air around thecondenser 234 or into air blown or drawn by the one or more fans 226that passes across the condenser 234, causing the working fluid 248 tocondense. As shown in FIG. 3A, the condenser 234 can be located betweenthe one or more fans 226 from the evaporator 232, but could also belocated on an opposite side of one or more of fans 226 (e.g., near anedge of the sub-assembly 210).

As shown in FIG. 3A, the condensate/vapor line 236 is at a slight(non-zero) angle so that gravity causes the condensed working fluid 248to flow back through the condensate/vapor lines 236 to the evaporator232. Thus, in some implementations, at least a portion of thecondensate/vapor line 236 is not parallel to the main surface of theframe 220. For example, the condenser-side end of the condensate/vaporline 236 can be about 1-5 mm, e.g., 2 mm, above the evaporator-side endof the condensate/vapor line 236. However, it is also possible for thecondensate/vapor line 236 to be horizontal tube, or even at a slightlynegative angle (although the positive angle provides an advantage ofgravity improving flow of the liquid from the condenser to theevaporator). Because there can be multiple heat generating electronicdevices on a single motherboard, there can be multiple evaporators onthe motherboard, where each evaporator corresponds to a singleelectronic device. As shown in FIG. 3A, there is a first evaporator 232and a second evaporator 232 as well as a first electronic device 224 anda second electronic device 224. The condensate/vapor line 236 connectingthe first evaporator to the second evaporator can be level.

During operation, the top surface of the working fluid 248 (as a liquid)inside the condenser 234 will be above the top surface liquid height ofthe working fluid 248 in the evaporator 232, e.g., by 1 to 10 mm. It canbe easier to achieve this with a condensate/vapor line 236 that is at aslight (positive non-zero) angle, but proper selection of the thermaland mechanical properties of the working fluid 248 in view of theexpected heat transport requirements for the thermosiphon system 230 maystill achieve this for a condensate/vapor line 236 that is horizontal orat a slightly negative angle. During operation, the liquid phase of aworking fluid 248 can fill a bottom portion of an interior volume of thecondensate/vapor line 236, with the bottom portion extending from thecondenser 234 to the evaporator 232, and a vapor phase of the workingfluid 248 can pass through a top portion of the interior volume of thecondensate/vapor line 236, with the top portion extending from thecondenser 234 to the evaporator 232.

In some implementations, the condenser 234 can be located at a heightabove the evaporator 232 such that a liquid phase of the working fluid248 fills a bottom portion of an interior volume of the condensate/vaporline 236 and such that during operation a top surface of the liquidphase has at a non-zero angle relative to horizontal from the condenser232 to the evaporator 234, and a vapor phase of the working fluid 248can pass through a top portion of the interior volume of thecondensate/vapor line 236, the top portion extending from the condenser234 to the evaporator 232.

FIGS. 3A-3B illustrate a thermosiphon system 230 with multipleevaporators 232; each evaporator 232 can contact a different electronicdevice 224, or multiple evaporators 232 could contact the sameelectronic device, e.g., if the electronic device is particularly largeor has multiple heat generating regions. The multiple evaporators 232can be connected by the condensate/vapor lines 236 to the condenser 234in series, e.g., a first condensate/vapor line connects the condenser234 to a first evaporator 232, and a second condensate/vapor line 236connects the first evaporator 232 to a second evaporator 232.Alternatively, some or all of the multiple evaporators 232 can beconnected by the condensate/vapor lines 236 to the condenser 234 inparallel, e.g., a first condensate/vapor line connects the condenser toa first evaporator, and a second condensate/vapor line connects thecondenser 234 to a second evaporator. Advantages of a serialimplementation may be fewer tubes, whereas an advantage of paralleltubes is that the tube diameters can be smaller.

FIGS. 3A-3B illustrate a thermosiphon system 230 in which a common lineis used for both the condensate flow from the condenser 234 to theevaporator 232 and for vapor flow from the evaporator 232 to thecondenser 234. Thus, in this implementation the fluid coupling betweenthe evaporator 232 and the condenser 234 consists of the combinedcondensate and vapor transfer line 236. In some implementations, therecan be separate lines for the vapor and the condensate. However, apotential advantage of the combined condensate and vapor transfer lineis that the line can be connected to a side of the condenser, reducingthe vertical height of the system relative to a system with a separateline for the vapor, since the vapor line is typically coupled to or nearthe top of the evaporator. The condensate/vapor line 236 can be aflexible tube or pipe, e.g., of copper or aluminum.

As shown in FIGS. 3A-3B, a controller 244 (or control system) iscommunicably coupled to one or more temperature sensors 246, one or morepressure/liquid level sensors 250 located in the evaporator 232, and apiston 252 mounted within a working volume 254 of the condenser 234, aswell as the one or more fans 226 (e.g., to control a speed or state ofthe fans 226). Generally, the controller 244 may receive one or moreinputs from the sensors 246 and/or sensors 250 (as well as other inputs)and control the piston 252 to adjust the working volume 254 of thecondenser 234 to, for example, better match a cooling capacity of thethermosiphon system 230 with a heat load of the electronic devices 224.

In some aspects, inputs into the controller 244, such as the sensors 246and/or sensors 250 may be indicative of the heat load of the electronicdevices 224. For example, the sensors 246 may measure a temperature ofthe electronic devices 224 and/or the motherboard 222. Also, sensors 250may measure a temperature, pressure, and/or liquid level of the workingfluid 248 in the evaporator 232. Although not specifically shown,temperature, pressure, and/or level of the working fluid 248 may bemeasured in the condenser 234 or conduit 236 as indicative of the heatload of the electronic devices 224.

One or more operational parameters of the electronic devices 224 mayalso be measured by sensors (not shown) that may be indicative of theheat load of the electronic devices 224. For example, power usage (e.g.,current, voltage, or power) of the electronic devices 224 may bemeasured and may be indicative of the heat load of the electronicdevices 224. As another example, operational speed or frequency (e.g.,Hz) of the electronic devices 224 may be measured and may be indicativeof the heat load of the electronic devices 224. As another example,utilization (e.g., number of jobs executed or to be executed, orotherwise) of the electronic devices 224 may be measured and may beindicative of the heat load of the electronic devices 224. Suchparameters may also be provided to the controller 244 and used to adjustthe piston 252.

The piston 252 may be adjusted (e.g., into and out of the condenser 232)by the controller 144 based, at least in part on the measured or sensedparameters described above. In some implementations, the piston 252 mayadjust the working volume 254 of the condenser 234. In some examples, asthe working volume 254 of the condenser 234 is decreased, a saturationpressure/temperature of the working fluid 248 in the condenser 234 isincreased, thereby increasing a cooling capacity of the thermosiphonsystem 230 (e.g., allowing a liquid level of the working fluid 248 toincrease in the evaporator 232). As the working volume 254 of thecondenser 234 is increased, the saturation pressure/temperature of theworking fluid 248 in the condenser 234 is decreased to decrease theliquid working fluid 248 in the evaporator 232, thereby decreasing acooling capacity of the thermosiphon system 230.

By adjusting the working volume 254 of the condenser 234 (therebyadjusting a liquid level of working fluid 248 in the evaporator 232),the cooling capacity of the thermosiphon system 230 may more closelymatch the heat load of the electronic devices 224 (e.g., indicated bythe one or more sensed or measure parameters described above). Bymatching or closely matching the heat load, the thermosiphon system 230may operate more efficiently, for example, operate closer to a dryoutcondition where all or most of the liquid working fluid 248 in theevaporator 232 is vaporized by the heat of the electronic devices 224.

In some implementations, the piston 252 may adjust a vibratory state ofthe condenser 234 (in addition to or alternatively with adjusting theworking volume 254). For example, based on a command from the controller244, the piston 252 may vibrate the condenser 234 in order to, forinstance, minimize a size of bubbles in the working fluid 248 enclosedwithin the condenser 234. By minimizing a size of the bubbles of theworking fluid 248 (e.g., breaking up larger bubbles into smallerbubbles), a thermal resistance of the working fluid 248 is decreased inthe condenser 234 (e.g., to condensing) and/or evaporator 232 (e.g., toboiling). As the thermal resistance to boiling/condensing is decreased,a heat transfer coefficient of the working fluid 248 is increased,thereby increasing a cooling capacity of the thermosiphon system 230.Thus, vibration of the condenser 234 by the piston 252 (or of theevaporator 232 in alternative implementations) may increase a coolingcapacity of the thermosiphon system 230 to match or more closely match aheat load of the electronic devices 224.

In some implementations of the illustrated thermosiphon system 230, thepiston 252 is actuated by a mechanical or electro-mechanical device(e.g., piston-cylinder, motor, or otherwise) that receives commands fromthe controller 244 to adjust the condenser 234 as described above. Inanother implementation, the controller 244 may be implemented as abimetallic member or phase change motor that is contactingly engagedwith the piston 252. Based on a temperature difference between thecomponents contacted by the bimetallic member/phase change motor, themember/motor may adjust a position of the piston 252 in the workingvolume 254 of the condenser 234. In this implementation, no externalpower and/or sensors (e.g., input to the controller 244) may benecessary for operation of the adjustable thermosiphon system 230,thereby decreasing complexity of the system 230.

FIGS. 4A-4B illustrate schematic side and top views, respectively, of aserver rack sub-assembly 310 that includes another exampleimplementation of a thermosiphon cooling system 330. The server racksub-assembly 310 includes a frame or cage 320, a printed circuit board322, e.g., motherboard, supported on the frame 320, one or moreheat-generating electronic devices 324, e.g., a processor or memory,mounted on the printed circuit board 322, and a thermosiphon system 330.One or more fans 326 can also be mounted on the frame 320.

The frame 320 can include or simply be a flat structure onto which themotherboard 322 can be placed and mounted, so that the frame 320 can begrasped by technicians for moving the motherboard into place and holdingit in position within the rack 105. For example, the server racksub-assembly 310 may be mounted horizontally in the server rack 105 suchas by sliding the frame 320 into the slot 107 and over a pair of railsin the rack 105 on opposed sides of the server rack sub-assembly310—much like sliding a lunch tray into a cafeteria rack. Although FIGS.4A-4B illustrate the frame 320 extending below the motherboard 322, theframe can have other forms (e.g., by implementing it as a peripheralframe around the motherboard) or may be eliminated so that themotherboard itself is located in, e.g., slidably engages, the rack 105.In addition, although FIG. 4A illustrates the frame 320 as a flat plate,the frame 320 can include one or more side walls that project upwardlyfrom the edges of the flat plate, and the flat plate could be the floorof a closed-top or open-top box or cage.

The illustrated server rack sub-assembly 310 includes a printed circuitboard 322, e.g., a motherboard, on which a variety of components aremounted, including heat-generating electronic devices 324. Although onemotherboard 322 is illustrated as mounted on the frame 320, multiplemotherboards may be mounted on the frame 320, depending on the needs ofthe particular application. In some implementations, the one or morefans 326 can be placed on the frame 320 so that air enters at the frontedge (at the left hand side in FIGS. 4A-4B) of the server racksub-assembly 310, closer to the front of the rack 105 when thesub-assembly 310 is installed in the rack 105, flows (as illustrated)over the motherboard 322, over some of the heat generating components onthe motherboard 322, and is exhausted from the server rack assembly 310at the back edge (at the right hand side), closer to the back of therack 105 when the sub-assembly 310 is installed in the rack 105. The oneor more fans 326 can be secured to the frame 320 by brackets. Thus, thefans 326 can pull air from within the frame 320 area and push the airafter it has been warmed out the rack 105. An underside of themotherboard 322 can be separated from the frame 320 by a gap.

The thermosiphon system 330 includes an evaporator 332, a condenser 334,and a condensate line 338 and a vapor line 336 that connect theevaporator 332 to the condenser 334. Thus, in this implementation, thereare separate conduits to transport liquid working fluid 348 from thecondenser 334 to the evaporator 332, and vapor working fluid 348 fromthe evaporator 332 to the condenser 334. One or both of the lines 338and 336 may be a flexible conduit, or may be a rigid conduit (e.g.,copper or aluminum).

The evaporator 332 contacts the electronic device 324 so that heat isdrawn by conductive heat transfer from the electronic device 324 to theevaporator 332. For example, the evaporator 332 is in conductive thermalcontact with the electronic device 324. In particular, the bottom of theevaporator 332 contacts the top of the electronic device 324. Inoperation, heat from the electronic device 324 causes a working fluid348 in the evaporator 332 to evaporate. The vapor then passes throughthe vapor line 336 to the condenser 334. Heat is radiated away from thecondenser 334, e.g., into air around the condenser 334 or into air blownor drawn by the one or more fans 326 that passes across the condenser334, causing the working fluid 348 to condense. As shown in FIG. 4A, thecondenser 334 can be located between the one or more fans 326 from theevaporator 332, but could also be located on an opposite side of one ormore of fans 326 (e.g., near an edge of the sub-assembly 310).

As shown in FIG. 4A, the vapor/condensate lines 336/338 are at a slight(non-zero) angle so that gravity causes the condensed working fluid 348to flow back through the condensate line 338 to the evaporator 332.Thus, in some implementations, at least a portion of each of thevapor/condensate lines 336/338 is not parallel to the main surface ofthe frame 320. For example, the condensate line 338 can be about 1-5 mm,e.g., 2 mm, above the vapor line 336. However, it is also possible forthe vapor/condensate lines 336/338 to be horizontal tube, or even at aslightly negative angle (although the positive angle provides anadvantage of gravity improving flow of the liquid from the condenser tothe evaporator). Because there can be multiple heat generatingelectronic devices on a single motherboard, there can be multipleevaporators on the motherboard, where each evaporator corresponds to asingle electronic device. As shown in FIG. 4A, there is a firstevaporator 332 and a second evaporator 332 as well as a first electronicdevice 324 and a second electronic device 324. The vapor/condensatelines 336/338 connecting the first evaporator to the second evaporatorcan be level.

During operation, the top surface of the working fluid 348 (as a liquid)inside the condenser 334 will be above the top surface liquid height ofthe working fluid 348 in the evaporator 332, e.g., by 1 to 10 mm. It canbe easier to achieve this with a condensate line 338 that is at a slight(positive non-zero) angle, but proper selection of the thermal andmechanical properties of the working fluid 348 in view of the expectedheat transport requirements for the thermosiphon system 330 may stillachieve this for a condensate line 338 that is horizontal or at aslightly negative angle.

FIGS. 4A-4B illustrate a thermosiphon system 330 with multipleevaporators 332; each evaporator 332 can contact a different electronicdevice 324, or multiple evaporators 332 could contact the sameelectronic device, e.g., if the electronic device is particularly largeor has multiple heat generating regions. The multiple evaporators 332can be connected by the vapor/condensate lines 336/338 to the condenser334 in series, e.g., a first set of vapor/condensate lines 336/338connect the condenser 334 to a first evaporator 332, and a second set ofvapor/condensate lines 336/338 connect the first evaporator 332 to asecond evaporator 332. Alternatively, some or all of the multipleevaporators 332 can be connected by the vapor/condensate lines 336/338to the condenser 334 in parallel, e.g., a first set of vapor/condensatelines 336/338 connect the condenser to a first evaporator, and a secondset of vapor/condensate lines 336/338 connect the condenser 334 to asecond evaporator. Advantages of a serial implementation may be fewertubes, whereas an advantage of parallel tubes is that the tube diameterscan be smaller.

As shown in FIGS. 4A-4B, a controller 344 (or control system) iscommunicably coupled to one or more temperature sensors 346, one or morepressure/liquid level sensors 350 located in the evaporator 332, and avalve 342 mounted within the condensate line 338, as well as the one ormore fans 326 (e.g., to control a speed or state of the fans 326).Generally, the controller 344 may receive one or more inputs from thesensors 346 and/or sensors 350 (as well as other inputs) and control thevalve 342, based at least in part on one or more of such inputs (orother inputs), to adjust the valve 342 to control an amount of liquidworking fluid 348 that flows to the evaporator 332 to, for example,better match a heat load of the electronic devices 324.

In alternative aspects, the valve 342 may be mounted in the vapor line336, in both vapor/condensate lines 336/338, or in a portion of thethermosiphon system 330 that is fluidly coupled to one or both of thevapor/condensate lines 336/338. In short, the valve 342 may bepositioned in any appropriate location to control the liquid level ofthe working fluid 348 in the evaporator 332.

In some aspects, inputs into the controller 344, such as the sensors 346and/or sensors 350 may be indicative of the heat load of the electronicdevices 324. For example, the sensors 346 may measure a temperature ofthe electronic devices 324 and/or the motherboard 322. Also, sensors 350may measure a temperature, pressure, and/or liquid level of the workingfluid 348 in the evaporator 332. Although not specifically shown,temperature, pressure, and/or level of the working fluid 348 may bemeasured in the condenser 334 or vapor/condensate lines 336/338 asindicative of the heat load of the electronic devices 324.

One or more operational parameters of the electronic devices 324 mayalso be measured by sensors (not shown) that may be indicative of theheat load of the electronic devices 324. For example, power usage (e.g.,current, voltage, or power) of the electronic devices 324 may bemeasured and may be indicative of the heat load of the electronicdevices 324. As another example, operational speed or frequency (e.g.,Hz) of the electronic devices 324 may be measured and may be indicativeof the heat load of the electronic devices 324. As another example,utilization (e.g., number of jobs executed or to be executed, orotherwise) of the electronic devices 324 may be measured and may beindicative of the heat load of the electronic devices 324. Suchparameters may also be provided to the controller 344 and used to adjustthe valve 342.

The valve 342 may be modulated (e.g. opened or closed) by the controller344 based, at least in part on the measured or sensed parametersdescribed above. In some implementations, by modulating the valve 342, aliquid level of the working fluid 348 in the evaporator 332 may beadjusted. By adjusting a liquid level of working fluid 348 in theevaporator 332, the cooling capacity of the thermosiphon system 330 maymore closely match the heat load of the electronic devices 324 (e.g.,indicated by the one or more sensed or measure parameters describedabove). By matching or closely matching the heat load, the thermosiphonsystem 330 may operate more efficiently, for example, operate closer toa dryout condition where all or most of the liquid working fluid 348 inthe evaporator 332 is vaporized by the heat of the electronic devices324.

FIG. 5 illustrates a schematic side view of a server rack sub-assembly510 that includes another example implementation of a thermosiphoncooling system 530. Thermosiphon system 530, as shown, may include oneor more components of the previously-described thermosiphon systems 130,230, and 330, as well as additional components. Each of the componentsmay, based at least in part on one or more sensed or measured parametersthat are indicative of a heat load and/or power usage of one or moreheat-generating electronic devices 524, control a liquid level of aworking fluid 548 in an evaporator 532 of the thermosiphon system 530.By controlling the liquid level of the working fluid 548, a coolingcapacity of the thermosiphon system 530 may more closely match a heatload of the one or more heat-generating electronic devices 524, therebyallowing the thermosiphon system 530 to operate more efficiently (e.g.,operate at or close to a dryout capacity) at many different heat loadsof the one or more heat-generating electronic devices 524.

The server rack sub-assembly 510 includes a frame or cage 520, a printedcircuit board 522, e.g., motherboard, supported on the frame 520, one ormore heat-generating electronic devices 524, e.g., a processor ormemory, mounted on the printed circuit board 522, and a thermosiphonsystem 530. One or more fans 526 can also be mounted on the frame 520 tocirculate air over the condenser 534, which in the illustratedimplementation, includes a heat transfer surface 560 (e.g., fins orother surface) mounted thereon.

The frame 520 can include or simply be a flat structure onto which themotherboard 522 can be placed and mounted, so that the frame 520 can begrasped by technicians for moving the motherboard into place and holdingit in position within the rack 105. For example, the server racksub-assembly 510 may be mounted horizontally in the server rack 105 suchas by sliding the frame 520 into the slot 107 and over a pair of railsin the rack 105 on opposed sides of the server rack sub-assembly510—much like sliding a lunch tray into a cafeteria rack. Although FIG.5 illustrates the frame 520 extending below the motherboard 522, theframe can have other forms (e.g., by implementing it as a peripheralframe around the motherboard) or may be eliminated so that themotherboard itself is located in, e.g., slidably engages, the rack 105.In addition, although FIG. 5 illustrates the frame 520 as a flat plate,the frame 520 can include one or more side walls that project upwardlyfrom the edges of the flat plate, and the flat plate could be the floorof a closed-top or open-top box or cage.

The illustrated server rack sub-assembly 510 includes a printed circuitboard 522, e.g., a motherboard, on which a variety of components aremounted, including heat-generating electronic devices 524. Although onemotherboard 522 is illustrated as mounted on the frame 520, multiplemotherboards may be mounted on the frame 520, depending on the needs ofthe particular application. In some implementations, the one or morefans 526 can be placed on the frame 520 so that air enters at the frontedge (at the left hand side in FIG. 5) of the server rack sub-assembly510, closer to the front of the rack 105 when the sub-assembly 510 isinstalled in the rack 105, flows (as illustrated) over the motherboard522, over some of the heat generating components on the motherboard 522,and is exhausted from the server rack assembly 510 at the back edge (atthe right hand side), closer to the back of the rack 105 when thesub-assembly 510 is installed in the rack 105. The one or more fans 526can be secured to the frame 520 by brackets. Thus, the fans 526 can pullair from within the frame 520 area and push the air after it has beenwarmed out the rack 105. An underside of the motherboard 522 can beseparated from the frame 520 by a gap.

The thermosiphon system 530 includes the evaporator 532, a condenser534, and a condensate line 538 and a vapor line 536 that connect theevaporator 532 to the condenser 534. Thus, in this implementation, thereare separate conduits to transport liquid working fluid 548 from thecondenser 534 to the evaporator 532, and vapor working fluid 548 fromthe evaporator 532 to the condenser 534. One or both of the lines 538and 536 may be a flexible conduit, or may be a rigid conduit (e.g.,copper or aluminum).

The evaporator 532 contacts the electronic device 524 so that heat isdrawn by conductive heat transfer from the electronic device 524 to theevaporator 532. For example, the evaporator 532 is in conductive thermalcontact with the electronic device 524. In particular, the bottom of theevaporator 532 contacts the top of the electronic device 524. Inoperation, heat from the electronic device 524 causes a working fluid548 in the evaporator 532 to evaporate. The vapor then passes throughthe vapor line 536 to the condenser 534. Heat is radiated away from thecondenser 534, e.g., into air around the condenser 534 or into air blownor drawn by the one or more fans 526 that passes across the condenser534, causing the working fluid 548 to condense. As shown in FIG. 5, thecondenser 534 can be located between the one or more fans 526 from theevaporator 532, but could also be located on an opposite side of one ormore of fans 526 (e.g., near an edge of the sub-assembly 510).

As shown in FIG. 5, the vapor/condensate lines 536/538 are at a slight(non-zero) angle so that gravity causes the condensed working fluid 548to flow back through the condensate line 538 to the evaporator 532.Thus, in some implementations, at least a portion of each of thevapor/condensate lines 536/538 is not parallel to the main surface ofthe frame 520. For example, the condensate line 538 can be about 1-5 mm,e.g., 2 mm, above the vapor line 536. However, it is also possible forthe vapor/condensate lines 536/538 to be horizontal tube, or even at aslightly negative angle (although the positive angle provides anadvantage of gravity improving flow of the liquid from the condenser tothe evaporator). Because there can be multiple heat generatingelectronic devices on a single motherboard, there can be multipleevaporators on the motherboard, where each evaporator corresponds to asingle electronic device. As shown in FIG. 5, there is a firstevaporator 532 and a second evaporator 532 as well as a first electronicdevice 524 and a second electronic device 524. The vapor/condensatelines 536/538 connecting the first evaporator to the second evaporatorcan be level.

During operation, the top surface of the working fluid 548 (as a liquid)inside the condenser 534 will be above the top surface liquid height ofthe working fluid 548 in the evaporator 532, e.g., by 1 to 10 mm. It canbe easier to achieve this with a condensate line 538 that is at a slight(positive non-zero) angle, but proper selection of the thermal andmechanical properties of the working fluid 548 in view of the expectedheat transport requirements for the thermosiphon system 530 may stillachieve this for a condensate line 538 that is horizontal or at aslightly negative angle.

FIG. 5 illustrates a thermosiphon system 530 with multiple evaporators532; each evaporator 532 can contact a different electronic device 524,or multiple evaporators 532 could contact the same electronic device,e.g., if the electronic device is particularly large or has multipleheat generating regions. The multiple evaporators 532 can be connectedby the vapor/condensate lines 536/538 to the condenser 534 in series,e.g., a first set of vapor/condensate lines 536/538 connect thecondenser 534 to a first evaporator 532, and a second set ofvapor/condensate lines 536/538 connect the first evaporator 532 to asecond evaporator 532. Alternatively, some or all of the multipleevaporators 532 can be connected by the vapor/condensate lines 536/538to the condenser 534 in parallel, e.g., a first set of vapor/condensatelines 536/538 connect the condenser to a first evaporator, and a secondset of vapor/condensate lines 536/538 connect the condenser 534 to asecond evaporator. Advantages of a serial implementation may be fewertubes, whereas an advantage of parallel tubes is that the tube diameterscan be smaller.

The thermosiphon system 530 may include one or more components that,based at least in part on one or more sensed or measured parameters thatare indicative of a heat load and/or power usage of the heat-generatingelectronic devices 524, control a liquid level of the working fluid 548in the evaporator 532 of the thermosiphon system 530.

For example, thermosiphon system 530 may include an actuator 542.Actuator 542 may be similar to the actuator 142 described above. Forinstance, the actuator 542 may be adjusted by a controller of thethermosiphon system 530 to adjust a height of the condenser 534 abovethe frame 520, which may also adjust a relative vertical distancebetween the condenser 534 and the evaporator 532. As the relativevertical distance between the condenser 534 and the evaporator 532increases, more liquid working fluid 548 flows to the evaporator 532,thereby increasing a cooling capacity of the thermosiphon system 530(e.g., allowing a liquid level of the working fluid 548 to increase inthe evaporator 532). The actuator 542 may also adjust an angularposition of the condenser 534 relative to the frame 520, for example, byrotating and/or pivoting the condenser 534. In some examples, as thecondenser 534 is rotated or pivoted, more liquid working fluid 548 mayflow to the evaporator 532, thereby increasing a cooling capacity of thethermosiphon system 530 (e.g., allowing a liquid level of the workingfluid 548 to increase in the evaporator 532). In some implementations,the actuator 542 may also adjust a vibratory state of the condenser 534(in addition to or alternatively with adjusting a height or angularposition). The actuator 542 may vibrate the condenser 534 in order to,for instance, minimize a size of bubbles in the working fluid 548enclosed within the condenser 534. By minimizing a size of the bubblesof the working fluid 548 (e.g., breaking up larger bubbles into smallerbubbles), a thermal resistance of the working fluid 548 is decreased inthe condenser 534 (e.g., to condensing) and/or evaporator (e.g., toboiling). As the thermal resistance to boiling/condensing is decreased,a heat transfer coefficient of the working fluid 548 is increased,thereby increasing a cooling capacity of the thermosiphon system 530.Thus, vibration of the condenser 534 by the actuator 542 (or of theevaporator 532 in alternative implementations) may increase a coolingcapacity of the thermosiphon system 530 to match or more closely match aheat load of the electronic devices 524.

The thermosiphon system 530 may include a piston 552. The piston 552 maybe similar to the piston 252 shown in FIGS. 3A-3B. For example, thepiston 552 may adjust a working volume 554 of the condenser 534. In someexamples, as the working volume 554 of the condenser 534 is decreased, asaturation pressure/temperature of the working fluid 548 in thecondenser 534 is increased, thereby increasing a cooling capacity of thethermosiphon system 530 (e.g., allowing a liquid level of the workingfluid 548 to increase in the evaporator 532). As the working volume 554of the condenser 534 is increased, the saturation pressure/temperatureof the working fluid 548 in the condenser 534 is decreased to decreasethe liquid working fluid 548 in the evaporator 532, thereby decreasing acooling capacity of the thermosiphon system 530.

By adjusting the working volume 554 of the condenser 534 (therebyadjusting a liquid level of working fluid 548 in the evaporator 532),the cooling capacity of the thermosiphon system 530 may more closelymatch the heat load of the electronic devices 524. By matching orclosely matching the heat load, the thermosiphon system 530 may operatemore efficiently, for example, operate closer to a dryout conditionwhere all or most of the liquid working fluid 548 in the evaporator 532is vaporized by the heat of the electronic devices 524. Further, thepiston 552 may adjust a vibratory state of the condenser 534 (inaddition to or alternatively with adjusting the working volume 554). Asdescribed above, vibration of the condenser 534 by the piston 552 (or ofthe evaporator 532 in alternative implementations) may increase acooling capacity of the thermosiphon system 530 to match or more closelymatch a heat load of the electronic devices 524.

The thermosiphon system 530 may include a valve 541. The valve 541 maybe similar to the valve 342 shown in FIGS. 4A-4B. For example, the valve541 may be modulated (e.g. opened or closed) based, at least in part onmeasured or sensed parameters indicative of a heat load and/or powerusage of the heat generating devices 524. In some implementations, bymodulating the valve 541, a liquid level of the working fluid 548 in theevaporator 532 may be adjusted. By adjusting a liquid level of workingfluid 548 in the evaporator 532, the cooling capacity of thethermosiphon system 530 may more closely match the heat load of theelectronic devices 524.

Thermosiphon system 530 may also include, as shown, a wick material 564,shown in FIG. 5 in the condensate line 538. In some aspects, the wickmaterial 564 may be selected based in part on an expected maximum heatload and/or power usage of the electronic devices 524, an expectedactual (or average) heat load and/or power usage of the electronicdevices 524, and/or other criteria. The wick material 564, in someaspects, may absorb a portion of the liquid working fluid 548 returnedto the evaporator 532. By absorbing a portion of the liquid, therebyslowing a rate of flow of the liquid working fluid 548 to theevaporator, a liquid level of the working fluid 548 in the evaporator532 may be controlled, thereby controlling a cooling capacity of thethermosiphon system 530.

As shown in FIG. 5, a controller (or control system) 544 may becommunicably coupled to one or more temperature sensors (not shown, butsimilar or identical to sensors 146 shown on FIG. 2A), one or morepressure/liquid level sensors (not shown, but similar or identical tosensors 150 shown on FIG. 2A) located in the evaporator 532, one or morefans 526 (e.g., to control a speed or state of the fans 526), as well asother components, such as the actuator 542, the piston 552, and/or thevalve 541. In some aspects, inputs into the controller 544 may beindicative of the heat load of the electronic devices 524. For example,the sensors may measure a temperature of the electronic devices 524and/or the motherboard 522. Also, sensors may measure a temperature,pressure, and/or liquid level of the working fluid 548 in the evaporator532. Although not specifically shown, temperature, pressure, and/orlevel of the working fluid 548 may be measured in the condenser 534 orvapor/condensate lines 536/538 as indicative of the heat load of theelectronic devices 524. One or more operational parameters of theelectronic devices 524 may also be measured by sensors that may beindicative of the heat load of the electronic devices 524. For example,power usage (e.g., current, voltage, or power) of the electronic devices524 may be measured and may be indicative of the heat load of theelectronic devices 524. As another example, operational speed orfrequency (e.g., Hz) of the electronic devices 524 may be measured andmay be indicative of the heat load of the electronic devices 524. Asanother example, utilization (e.g., number of jobs executed or to beexecuted, or otherwise) of the electronic devices 524 may be measuredand may be indicative of the heat load of the electronic devices 524.

Generally, the controller 544 may receive one or more inputs fromsensors that sense or measure parameters associated with a heat loadand/or power usage of the electronic devices. The controller 544 may usethe sensed or measured parameters to control the actuator 542, thepiston 552, and/or the valve 541 to control an amount of liquid workingfluid 548 that flows to the evaporator 532 to, for example, better matcha heat load of the electronic devices 524. In some aspects, thecontroller 544 may select one of the actuator 542, the piston 552, orthe valve 541 to adjust the amount of liquid working fluid 548 thatflows to the evaporator 532 based on, for instance, the sensed,measured, or determined heat load and/or power usage of the electronicdevices 524.

In alternative aspects, the valve 541 may be mounted in the vapor line536, in both vapor/condensate lines 536/538, or in a portion of thethermosiphon system 530 that is fluidly coupled to one or both of thevapor/condensate lines 536/538. In short, the valve 541 may bepositioned in any appropriate location to control the liquid level ofthe working fluid 548 in the evaporator 532.

FIGS. 6A-6B illustrate schematic side and top views of a portion 600 ofa thermosiphon cooling system, such as the thermosiphon systems 130,230, 330, and/or 530. In some aspects, the portion 600 may be anevaporator 600 of a thermosiphon system as described above. Asillustrated, the evaporator 600 includes a chamber 646 and heat transfersurfaces 642. The evaporator 600 includes a base 640 and a case 644 thatis secured to the base 640. The case 644, in some aspects, can beprovided by a tube of the condensate/vapor line (e.g., condensate/vaporline 136). A volume sealed above the base 640 inside the case 644provides a chamber 646 for the evaporator 600. The top surface of thebase 640 provides an evaporator pan. That is, the top surface of thebase 640 includes a portion i) that is recessed relative to passages 639in the case 644, and ii) in which a liquid phase of the working fluid660 collects.

As illustrated, the heat transfer surfaces 642 project upwardly from theevaporator base 640 so that they are above the bottom of the case 640.When a liquid phase of the working fluid 660 overflows the evaporatorpan of the base 640, it floods the bottom section of the chamber 646defined by the case 640. Thus, the bottom of chamber 646 defined by thecase 640 can be considered a floodplain. In addition, this ensures thatthe heat transfer surfaces 642 remain only partially submerged in theliquid phase of the working fluid 660.

The base 640 can be formed of the same material as the case 640, e.g.,aluminum, or may be formed from a different thermally conductivematerial, e.g., copper. The bottom of the base 640 can directly contactthe electronic device 124, e.g., the top surface of the electronicdevice 124. Alternatively, the bottom of the base 640, can be connectedto the electronic device 124, e.g., the top surface of the electronicdevice 124, by a thermally conductive interface material, e.g., athermally conductive pad or layer, e.g., a thermally conductive greaseor adhesive.

The heat transfer surfaces 642, as shown, includes a plurality of fins650 that contact the bottom interior surface of the housing, e.g., thetop surface of the base 640. The evaporator fins 650 project upwardlyfrom the pan of the evaporator base 640. Thus, the evaporator fins 650provide a thermally conductive area that transfers heat from the base640 to the working fluid 660. The tops of the fins 650 can project intothe chamber 646 and can be arranged substantially in parallel rows, asshown. In some implementations, the fins 650 extend generally parallelto the width of the chamber 646.

In addition, the evaporator fins 650 can be configured to draw theworking fluid 660 away from the base 640 by capillary action. Forexample, the evaporator fins 650 can be stamped or otherwise imprintedwith features, e.g., grooving, which tends to draw the working fluidupward. In some implementations, the fins 650 can have undulations alongtheir length. The undulations can have a pitch between 1 mm and 2 mm andan amplitude between 0.1 and 0.5 mm. These undulations can cause some ofthe liquid phase of the working fluid 660 to move up the fins 650 bycapillary action. This can improve the efficiency of the evaporator 600by exposing more of the surface area of the fins 650 to the liquid phaseof the working fluid. The fins 650 can be constructed of the samematerial as the evaporator, e.g., aluminum. Alternatively, the fins 650can be constructed of a different thermally conductive material, e.g.,copper.

In the illustrated implementation, the fins 650 are integrally formed inthe evaporator base 640 and thus, the base 640 and fins 650 are formedfrom a single material piece (e.g., copper or aluminum or otherthermally conductive material). In some aspects, the integral nature ofthe fins 650 and the base 640, e.g., in contrast to separate fins or finstacks that are brazed or otherwise connected to the evaporator base,may reduce a complexity of forming the evaporator 600. For example, thefins 650 may be formed with higher density and tighter tolerancesrelative to fins or fin stacks that are brazed or otherwise connected tothe evaporator base 640. The higher density and tighter tolerances ofthe illustrated design may provide for improved heat transferperformance.

FIGS. 7-8 are flowcharts that illustrate example methods of cooling heatgenerating devices in a data center with a thermosiphon cooling system.Turning to method 700 shown in FIG. 7, this method may begin at step 702by circulating a working fluid between an evaporator and a condenser ofa thermosiphon that is mounted on a server tray sub-assembly. Theworking fluid may circulate from the evaporator to the condenser as avapor. The working fluid may circulate (e.g., by gravity) from thecondenser to the evaporator as a liquid.

Step 704 includes cooling the motherboard mounted electronic deviceswith the thermosiphon. For instance, heat energy is transferred from theone or more motherboard mounted electronic devices to a liquid workingfluid in the evaporator (e.g., to vaporize the liquid). The heat energyis then moved, with the vaporized working fluid, to the condenser andreleased from the working fluid (e.g., to ambient air or airflow aroundthe condenser). The working fluid condenses in the condenser to aliquid, which moves back to the evaporator.

Step 706 includes measuring a parameter associated with heat generated,or power used, by the electronic devices. The parameter may be sensed ormeasured and may directly sense/measure heat generated or power used bythe electronic devices. The parameter may also indirectly sense/measureheat generated or power used by the electronic devices. For instance,the parameter may be a temperature of: the sub-assembly, a motherboardof the sub-assembly, one or more of the electronic devices, thethermosiphon, an air or airflow around the electronic devices orthermosiphon, or the working fluid. The parameter may be a pressure ofthe working fluid. The parameter may be a level of liquid working fluidin the evaporator. The parameter may also be a measured, estimated,predicated, or nameplate power (e.g., current, voltage, and/or wattage)usage of the one or more electronic devices. The parameter may also be ameasured, estimated, or predicated utilization of the one or moreelectronic devices.

Step 708 includes adjusting a liquid level of the working fluid in theevaporator based, at least in part, on the measured or sensed parameter.In some aspects, an example implementation of step 708 may be performedaccording to method 800 shown in FIG. 8. For example, step 802 includesoperating an actuator coupled to the thermosiphon. In some aspects, theactuator may be coupled to the condenser. For instance, in some aspects,the actuator may include a piston mounted within a working volume of thecondenser, or a height adjustment assembly coupled to the condenser, ora vibration assembly coupled to the condenser. In some aspects, theactuator may include a valve positioned within a condensate line thatreturns liquid working fluid to the evaporator from the condenser.

Step 804 includes adjusting a portion of the thermosiphon with theactuator. For example, the actuator may adjust a vertical distancebetween the condenser and the evaporator (e.g., adjust a height of thecondenser from the sub-assembly). In some aspects, the actuator mayadjust a working volume of the condenser (e.g., with the piston). Insome aspects, the actuator may adjust a flow of the liquid working fluidfrom the condenser to the evaporator (e.g., with a valve, wick materialor otherwise). In some aspects, the actuator may vibrate the condenseror other portion of the thermosiphon in order to reduce a bubble size ofthe working fluid in the thermosiphon.

Step 806 includes adjusting a flow of liquid of the working fluid to theevaporator or flow of vapor to the condenser based on adjustment of thethermosiphon. Step 808 includes adjusting the liquid level of theworking fluid in the evaporator based on adjustment of the flow ofliquid of the working fluid to the evaporator or flow of vapor to thecondenser. Step 810 includes matching a cooling capacity of thethermosiphon with a heat load of the electronic devices based onadjusting the liquid level of the working fluid in the evaporator.

In some implementations, matching (exactly or almost) the coolingcapacity of the thermosiphon with the heat load of the electronicdevices may allow for more efficient operation of the thermosiphon. Forinstance, a thermosiphon might operate most efficiently at a designcooling capacity, e.g., a dryout capacity where all or substantially allof the liquid in the evaporator is vaporized by heat energy transferredfrom the electronic devices. Such a design cooling capacity may bechosen, and indeed may be required, based on a maximum heat load of theelectronic devices (e.g., corresponding to a maximum or nameplate powerusage). However, since the electronic devices may not typically operate,and may never operate, at the nameplate power usage, the heat load ofthe electronic devices (e.g., average or actual) may be less than themaximum heat load. The design cooling capacity of the thermosiphon maytherefore be too large, causing the thermosiphon to typically operate ina less efficient (or inefficient) state (e.g., with buildup of liquidworking fluid in the evaporator). Operation of the actuator to adjustthe thermosiphon to match (exactly or closely) a dynamic heat load ofthe electronic devices may thus allow the thermosiphon to be chosen fora maximum heat load (e.g., as the electronic devices operate atnameplate power) while still allowing for the most efficient operationat various heat loads less than the maximum heat load.

FIG. 9 is a schematic diagram of a control system (or controller) 900.The system 900 can be used for the operations described in associationwith any of the computer-implemented methods described previously, forexample as or as part of the controllers 144/244/344 or othercontrollers described herein. For example, the system 900 may be used inproviding local control for particular ones of or small groups of,combined power/cooling units described above, or in providing mastercontrol over an entire data center or multiple data centers of suchunits. Moreover, the system 900 may describe computing resources thatmay operate as the loads to be cooled by the systems and methodsdescribed above.

The system 900 is intended to include various forms of digitalcomputers, such as laptops, desktops, workstations, personal digitalassistants, servers, blade servers, mainframes, and other appropriatecomputers. The system 900 can also include mobile devices, such aspersonal digital assistants, cellular telephones, smartphones, and othersimilar computing devices. Additionally the system can include portablestorage media, such as, Universal Serial Bus (USB) flash drives. Forexample, the USB flash drives may store operating systems and otherapplications. The USB flash drives can include input/output components,such as a wireless transmitter or USB connector that may be insertedinto a USB port of another computing device.

The system 900 includes a processor 910, a memory 920, a storage device930, and an input/output device 940. Each of the components 910, 920,930, and 940 are interconnected using a system bus 950. The processor910 is capable of processing instructions for execution within thesystem 900. The processor may be designed using any of a number ofarchitectures. For example, the processor 910 may be a CISC (ComplexInstruction Set Computers) processor, a RISC (Reduced Instruction SetComputer) processor, or a MISC (Minimal Instruction Set Computer)processor.

In one implementation, the processor 910 is a single-threaded processor.In another implementation, the processor 910 is a multi-threadedprocessor. The processor 910 is capable of processing instructionsstored in the memory 920 or on the storage device 930 to displaygraphical information for a user interface on the input/output device940.

The memory 920 stores information within the system 900. In oneimplementation, the memory 920 is a computer-readable medium. In oneimplementation, the memory 920 is a volatile memory unit. In anotherimplementation, the memory 920 is a non-volatile memory unit.

The storage device 930 is capable of providing mass storage for thesystem 900. In one implementation, the storage device 930 is acomputer-readable medium. In various different implementations, thestorage device 930 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device.

The input/output device 940 provides input/output operations for thesystem 900. In one implementation, the input/output device 940 includesa keyboard and/or pointing device. In another implementation, theinput/output device 940 includes a display unit for displaying graphicaluser interfaces.

The features described can be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them. The apparatus can be implemented in a computerprogram product tangibly embodied in an information carrier, e.g., in amachine-readable storage device for execution by a programmableprocessor; and method steps can be performed by a programmable processorexecuting a program of instructions to perform functions of thedescribed implementations by operating on input data and generatingoutput. The described features can be implemented advantageously in oneor more computer programs that are executable on a programmable systemincluding at least one programmable processor coupled to receive dataand instructions from, and to transmit data and instructions to, a datastorage system, at least one input device, and at least one outputdevice. A computer program is a set of instructions that can be used,directly or indirectly, in a computer to perform a certain activity orbring about a certain result. A computer program can be written in anyform of programming language, including compiled or interpretedlanguages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment.

Suitable processors for the execution of a program of instructionsinclude, by way of example, both general and special purposemicroprocessors, and the sole processor or one of multiple processors ofany kind of computer. Generally, a processor will receive instructionsand data from a read-only memory or a random access memory or both. Theessential elements of a computer are a processor for executinginstructions and one or more memories for storing instructions and data.Generally, a computer will also include, or be operatively coupled tocommunicate with, one or more mass storage devices for storing datafiles; such devices include magnetic disks, such as internal hard disksand removable disks; magneto-optical disks; and optical disks. Storagedevices suitable for tangibly embodying computer program instructionsand data include all forms of non-volatile memory, including by way ofexample semiconductor memory devices, such as EPROM, EEPROM, and flashmemory devices; magnetic disks such as internal hard disks and removabledisks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Theprocessor and the memory can be supplemented by, or incorporated in,ASICs (application-specific integrated circuits).

To provide for interaction with a user, the features can be implementedon a computer having a display device such as a CRT (cathode ray tube)or LCD (liquid crystal display) monitor for displaying information tothe user and a keyboard and a pointing device such as a mouse or atrackball by which the user can provide input to the computer.Additionally, such activities can be implemented via touchscreenflat-panel displays and other appropriate mechanisms.

The features can be implemented in a control system that includes aback-end component, such as a data server, or that includes a middlewarecomponent, such as an application server or an Internet server, or thatincludes a front-end component, such as a client computer having agraphical user interface or an Internet browser, or any combination ofthem. The components of the system can be connected by any form ormedium of digital data communication such as a communication network.Examples of communication networks include a local area network (“LAN”),a wide area network (“WAN”), peer-to-peer networks (having ad-hoc orstatic members), grid computing infrastructures, and the Internet.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made without departing fromthe spirit and scope of what is described. For example, the steps of theexemplary flow charts in FIGS. 7-8 may be performed in other orders,some steps may be removed, and other steps may be added. Further, insome implementations, a phase change material may be positioned, forexample, between an evaporator of a thermosiphon and one or more heatgenerating electronic devices to increase a thermal contact area betweenthe evaporator and the devices. Accordingly, other embodiments arewithin the scope of the following claims.

What is claimed is:
 1. A server tray sub-assembly comprises: amotherboard; a plurality of heat generating electronic devices mountedon the motherboard; a thermosiphon mounted on the motherboard, thethermosiphon comprising: an evaporator in heat transfer communicationwith the plurality of heat generating electronic devices; a condenserfluidly coupled to the evaporator with a fluid conduit that slopesdownward from the condenser to the evaporator; and a multi-phase fluidcontained in the thermosiphon; and a control system comprising: asensing device operable to sense a value associated with an amount ofheat generated by the plurality of heat generating electronic devices;and an actuator operatively coupled to the thermosiphon to adjust anamount of the multi-phase fluid in the evaporator based, at least inpart, on the sensed value.
 2. The server tray sub-assembly of claim 1,wherein the actuator is operatively coupled to the condenser and isconfigured to adjust the condenser based, at least in part, on thesensed value.
 3. The server tray sub-assembly of claim 2, wherein theactuator is configured to adjust at least one of a vertical distancebetween the condenser and the evaporator; a working volume of thecondenser; or an angular position of the condenser relative to themotherboard.
 4. The server tray sub-assembly of claim 2, wherein theactuator comprises a piston mounted in an interior volume of thecondenser.
 5. The server tray sub-assembly of claim 4, wherein thepiston is configured to adjust a working volume of the condenser based,at least in part, on the sensed value.
 6. The server tray sub-assemblyof claim 1, wherein the actuator is configured to vibrate the condenser.7. The server tray sub-assembly of claim 1, wherein the actuator isconfigured to adjust a flow of a liquid phase of the multi-phase fluidfrom the condenser to the evaporator based, at least in part, on thesensed value.
 8. The server tray sub-assembly of claim 1, wherein theactuator comprises a valve arranged in a liquid line of the fluidconduit.
 9. The server tray sub-assembly of claim 1, wherein theactuator comprises a wicking material mounted in a liquid line of thefluid conduit.
 10. The server tray sub-assembly of claim 9, wherein thefluid conduit is flexible.
 11. The server tray sub-assembly of claim 9,wherein the wicking material comprises an absorbent material.
 12. Theserver tray sub-assembly of claim 11, wherein the absorbent material isconfigured to absorb a portion of a liquid phase of the multi-phasefluid.
 13. The server tray sub-assembly of claim 1, wherein the sensedvalue comprises at least one of: a temperature of air adjacent theplurality of heat generating electronic devices, a temperature of airadjacent the condenser, a temperature of the plurality of heatgenerating electronic devices, a temperature of the motherboard, aliquid level of the multi-phase fluid in the evaporator, a pressure ofthe multi-phase fluid, a temperature of the multi-phase fluid, a powerusage of the plurality of heat generating electronic devices, afrequency of one or more of the plurality of heat generating electronicdevices, or a utilization of one or more of the plurality of heatgenerating electronic devices.
 14. The server tray sub-assembly of claim1, wherein the fluid conduit comprises copper or aluminum.
 15. Theserver tray sub-assembly of claim 1, further comprising a fan mounted tothe motherboard.
 16. The server tray sub-assembly of claim 15, whereinthe fan is configured to circulate airflow across the condenser.
 17. Theserver tray sub-assembly of claim 16, further comprising an enhancedheat transfer surface coupled to the condenser.
 18. The server traysub-assembly of claim 17, wherein the fan is positioned on themotherboard to circulate the airflow across the enhanced heat transfersurface coupled to the condenser.
 19. The server tray sub-assembly ofclaim 1, wherein the plurality of heat generating electronic devicescomprise at least one data center server device.
 20. The server traysub-assembly of claim 19, wherein the at least one data center serverdevice comprises a processor.